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14.  abstract  The  objective  is  to  demonstrate  that  interactive  data  visualizations  designed  to  transform  and  consolidate 
complex  multimodal  physiological  data  into  integrated  interactive  displays  will  reduce  clinician  cognitive  load  and  will  result  in 
reductions  in  medical  error  and  improvements  in  patient  care,  safety,  and  efficiency.  Research  accomplishments  include:  1) 
Devised  methods  to  both  filter  and  interpolate  physiological  data,  2)  Created  physiologic  indices  for  clinically  meaningful 
variation  among  several  parameters  using  a  version  of  empirical  orthogonal  functions,  3)  Applied  the  above  physiologic  indices 
to  create  a  patient  overview  visualization  to  help  clinicians  identify  patients  that  are  changing  and  inserted  these  indices  into 
the  sepsis  specific  decision  support  visualization,  4)  Created  a  sepsis  identification  visualization  tool  to  help  clinicians  identify 
patients  headed  for  septic  shock,  and  5)  Generated  a  visualization  tool  to  help  clinicians  assess  intracranial  pressure  and 
generate  patient  specific  physiologic  set-points  for  goal  directed  therapy,  and  6)  Demonstrate  a  real-time  data  feed  to  a 
visualization  tool  that  contains  elements  from  each  of  the  three  prototype  visualizations.  In  the  next  period  a  cognitive 
experiment  will  be  conducted  to  determine  the  effectiveness  of  these  visualization  tools. 
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INTRODUCTION 

Intensive  care  unit  clinicians  may  be  confronted  by  more  than  200  variables  for  critically-ill  patients  during 
rounds  each  morning.1  In  contrast  people  are  not  able  to  judge  the  degree  of  relatedness  between  more  than 
two  variables  without  assistance.2  A  late-2009  report  by  the  National  Research  Council  (NRC)  suggests  that 
information  technology  (IT)  solutions  for  medicine  should  shift  their  focus  from  current  solutions  that  provide 
cognitive  support  on  each  individual  piece  of  equipment  or  report  to  one  that  supports  clinicians  in  synthesizing 
massive  amounts  of  data  to  acquire  a  conceptual  model  of  the  whole  patient. 3  The  hypothesis  of  this  research 
effort  is  to  demonstrate  that  reducing  clinician  cognitive  load  by  consolidating  complex  multimodal  physiological 
information  into  coherent  images  using  data  visualization  will  result  in  reductions  in  medical  error  and 
improvements  in  patient  care,  safety,  and  efficiency.  Our  research  objective  is  to  design,  develop,  and  test  a 
prototype  data  visualization  system  that  integrates  and  presents  complex  disparate  data  streams  into  dynamic 
visualizations  that  enable  clinicians  to  more  easily  assimilate  the  information  and  derive  insight  from  it  resulting 
in  faster  more  accurate  treatment  decisions.  There  are  4  phases  to  this  research.  Phase  1  is  to  develop  a 
paper  prototype  interactive  visualization.  In  Phase  2  and  3  Columbia  University  investigators  focus  on  data 
acquisition  and  analysis  of  data  to  construct  patient  cases  for  the  purpose  of  the  visualization  experiments 
while  Draper  Laboratory  investigators  focus  on  developing  the  visualization  tools  detailed  in  Phase  1 .  During 
Phase  4  a  set  of  experiments  will  be  conducted  to  evaluate  the  effectiveness  of  the  visualization  against  an 
unaided  standard-of-care  condition. 
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BODY 

The  Statement  of  Work  on  this  project  is  for  Columbia  University  and  Draper  Laboratory  Investigators  to 
develop  novel  visualization  tools  that  represent  health  data  acquired  from  patients  in  the  neurological  intensive 
care  unit  of  Columbia  University  Medical  Center.  The  general  aim  of  this  effort  is  to  demonstrate  that 
interactive  data  visualizations  designed  to  transform  and  consolidate  complex  multimodal  physiological  data 
into  integrated  interactive  displays  will  reduce  clinician  cognitive  load  and  will  result  in  reductions  in  medical 
error  and  improvements  in  patient  care,  safety,  and  efficiency.  Three  case  scenarios  to  focus  on  were 
identified  in  the  scope  of  work.  The  first  focus  area  is  to  develop  a  visualization  to  help  clinicians  identify 
patients  that  are  changing  physiological  states  and  therefore  require  medical  attention.  We  proposed  applying 
empirical  orthogonal  function  (EOF)  analysis  to  identify  patient  states  and  variables  that  have  changed  over  a 
specified  time  period.  Once  the  clinician  is  alerted  to  a  change  in  patient  state,  decision  support  tools  should 
facilitate  their  clinical  investigation  and  in  making  a  specific  decision  that  fall  into  one  of  three  general 
categories:  1)  identify  a  physiologic  end  point  to  guide  goal-directed  therapy,  and  determine  the  relative 
effectiveness  of  a  specific  or  set  of  interventions  in  achieving  those  goals,  2)  determine  the  onset  of  a 
secondary  complication  and  intervene,  3)  identify  active  physiologic  processes  that  can  affect  overall  clinical 
management.  The  project  is  broken  down  into  four  key  deliverable  milestones  including:  1)  prototype 
visualization  development,  2)  data  collection,  analysis,  and  deployment  to  back-end  data  system,  3)  interactive 
visual  interface  complete  and  connected  with  the  data,  and  4)  conduct  cognitive  experiments  to  evaluate  the 
effectiveness  of  the  visualization  tools  against  an  unaided  condition. 

Draper  Laboratory  human-factor  engineers  gained  a  comprehensive  understanding  of  the  needs  and  data 
requirements  for  the  three  case  scenarios  by  completing  (3/15/13)  a  cognitive  work  analysis  (CWA)  with 
clinical  staff  from  the  Columbia  University  Neurological  Intensive  Care  Unit.  Columbia  University  Investigators 
and  Draper  Laboratory  human-factor  engineers  underwent  an  iterative  process  to  translate  the  CWA  results 
into  requirements  for  prototype  visualizations.  Mockup  visualizations  were  developed  for  an  integrated  system- 
level  overview  that  utilizes  empirical  orthogonal  functions  (complete  4/26/2013),  and  for  specific  decisions 
related  to  patient  development  of  sepsis  (complete  4/15/2013)  and  intracranial  pressure  management 
(complete  4/26/2013).  We  deployed  a  back-end  MongoDB  database  and  have  been  able  to  deposit  data  into 
it.  However,  we  found  that  building  an  interface  to  connecting  the  visualization  tool  directly  to  this  data  source 
was  not  realistic  given  the  time  constraints  and  need  for  experimenter  control  in  the  experiment.  Instead  we 
created  static  data  files  that  are  read  directly  for  the  purpose  of  the  experiment.  As  we  stated  in  our  quarter 
report  this  has  not  impacted  achieving  the  primary  goals  of  the  project.  Alternatively  however  we  have  been 
able  to  incorporate  elements  of  our  visualization  tools  into  a  web-based  visualization  tool  to  connect  to  a  real¬ 
time  data  stream  (Milestone  2)  as  will  be  detailed  later  in  this  report.  The  results  of  the  cognitive  work  analysis 
(reportable  outcome  1)  and  prototype  mockups  (reportable  outcome  2)  are  detailed  below.  Each  mockup 
underwent  a  series  of  design  iterations  to  ensure  that  the  visualizations  made  sense  for  the  tasks  and 
decisions  that  the  clinicians  need  to  solve. 

PATIENT  OVERVIEW  VISUALIZATION 

The  goal  of  the  patient  overview  visualization  is  to  identify  patient  state  changes  and  identifying  patients  that 
need  clinician  attention.  The  most  difficult  task  for  clinicians  is  not  determining  which  patients  are  doing  poorly 
-  this  is  usually  self-evident  -  it  is  determining  which  patients  are  changing  and  require  intervention.  We 
hypothesize  that  an  interactive  visualization  can  be  designed  to  support  clinicians  in  more  quickly  recognizing 
changes  to  patient  state,  situation  awareness,  and  faster  ability  to  pinpoint  emerging  situations.  Empirical 
orthogonal  function  analysis  is  a  multivariate  temporal  analysis  technique  that  can  be  used  to  identify  sources 
of  variance  in  patient  data  and  what  physiological  parameters  vary  together. 

During  the  second  quarter  we  set  out  to  determine  the  best  way  to  present  empirical  orthogonal  function  (EOF) 
results  to  clinicians  to  denote  patient  state  changes  and  identify  variables  accounting  for  the  variance.  Data 
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preprocessing  was  an  essential  first  step  to  applying  this  technique  and  we  addressed  two  characteristics  of 
the  data,  specially  outlier  removal  and  missing  data.  As  a  result  of  the  fact  that  the  physiological  data 

-  collected  in  the  Neurological  ICU 

Figure  1.  Unfiltered  (top)  versus  filtered  (bottom)  temperature  values  is  not  collected  in  a  controlled 

environment,  there  are  frequent 
outliers  that  represent  spurious 
data  from  the  perspective  of 
patient  care.  For  example, 
catheters  measuring  body 
temperature  are  often  removed 
prior  to  the  termination  of  data 
collection  leading  to  temperature 
readings  in  both  the  body 
temperature  range  as  well  as 
temperature  readings  in  the 
room  temperature  range.  We 
confronted  difficulties  leveraging 
existing  computational  tools 
given  the  high  levels  of 
missingness  in  our  datasets. 

We  explored  interpolation  and 
matrix  factorization  methods  for 
imputing  missing  values  in  the 
multidimensional  setting.  We  also  explored  custom  extensions  to  existing  matrix  factorization  methods  to 
adapt  them  to  time  series  and  potentially  identify  spurious  outliers  simultaneously.  The  best  procedure  we 
found  was  to  implement  a  variational  Bayesian  mixture  model  for  the  detection  and  removal  of  such  spurious 
states  -  the  results  of  which  applied  to  the  same  variable  (temperature)  can  be  seen  in  the  figure  1 . 

Empirical  orthogonal  function  (EOF)  analysis  is  designed  to  identify  and  highlight  the  set  of  variables  that  are 
contributing  most  to  the  variability  of  a  patient's  state  during  a  given  time  period.  4,5  This  analysis  can  be  used 
both  for  visualizing  the  patient's  state 
over  time,  but  can  also  be  used  to 
identify  points  of  significant  change. 

Practically,  due  to  the  fact  that  many 
physiological  variables  are  not  always 
recorded,  we  had  to  first  address  the 
issue  of  missingness.  Although,  we 
experimented  with  several  approaches, 
the  combination  of  outlier  removal  as 
described  above  and  constant 
interpolation  was  determined  to  be  the 
most  accurate  approach.  We  have 
performed  the  EOF  analysis  on  2  fully 
anonymized  patient  datasets.  A  brief 
period  from  one  of  these  patients  is 
displayed  in  figure  2  highlighting  a 
period  of  significant  change  where 
temperature  is  one  of  the  variables 
involved  in  this  change. 


Figure  2.  EOF  Analysis  (top)  and  temperature  (TMP-2;  bottom) 
values 
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Figure  3.  EOF  Analysis  Results  for  Individual  Patient 
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Several  steps  are  required  to  apply 
EOF  to  physiologic  data.  The  raw 
data  must  be  visualized  and 
preprocessed  on  a  per-variable 
basis  with  variational  Gaussian 
mixture  models  for  outlier  removal. 
Interpolation  is  then  used  to  create 
a  uniformly  sampled  time  series 
based  on  the  original  series.  Data 
is  separated  across  all  variables 
into  time  bins  and  for  each  bin  the 
primary  eigenvector  is  calculated. 
This  represents  the  unitized  linear 
combination  of  variables  that 
explains  the  most  variance  for  the 
bin.  Each  patient’s  primary 
eigenvector  for  each  time  bin  can 
be  visualized  on  a  heatmap  (See 
Figure  3.) 


We  quickly  discovered  that  while  heatmaps  accurately  represented  the  varying  relationships  among  all  the 
parameters  that  it  would  be  difficult  to  use  this  information  to  determine  if  the  patient  was  changing  in  ways  that 
required  attention.  This  is  a  key  point  of  learning  and  emphasis,  specifically  that  visualizing  relationships 
among  patient  data  parameters  accurately  does  not  in  itself  solve  the  real  problem  of  helping  clinicians  make 

better  decisions  from  patient  data.  At  best 


Figure  4.  EOF  Composite  Score  for  Individual  Patient 
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From  top  to  bottom:  The  variable  weightings  for  the  first  EOF, 
the  projection  of  the  first  EOF  onto  patient  2234's  multivariate 
time  series,  patient  2234's  heart  rate  time  series  and  patient 
2234's  blood  pressure  time  series. 


this  heatmap  requires  significant  effort  on 
part  of  the  users  to  extract  meaningful 
information  from  it.  Data  visualization  in 
itself  then  is  not  sufficient  and  we  thereafter 
added  the  requirement  that  an  analysis  must 
both  represent  the  relationships  among 
parameters  and  convert  this  information  into 
a  form  that  fits  into  the  mental  framework  of 
clinicians. 

In  the  second  and  third  quarter  we  worked  to 
determine  the  best  way  to  present  results  to 
clinicians  to  denote  patient  state  change  and 
to  highlight  what  variables  account  for  this 
variance.  We  expanded  our  approach  to 
include  an  alternative  method  of  applying 
EOF  analysis.  In  contrast  to  the  method  that 
we  previously  described,  this  method 
evaluated  the  directions  in  the  space  of 
measured  variables  that  explains  the  most 
variance  over  all  time  for  all  patients.  The 
initial  method  we  outlined  evaluates  a  similar 
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quantity  for  small  time  bins  in  a 
patient  specific  manner.  The  new 
technique  discovers  the  weighting 
on  the  measured  variables  that 
are  most  meaningful  for 
explaining  the  variation  from  time 
to  time  within  a  patient  and 
across  patients.  We  will  explore 
the  interpretability  of  these 
directions  in  the  space  of 
measured  variables  and  leverage 
these  discovered  directions  to 
create  composite  clinical  scores 
for  each  dimension.  These 
composite  clinical  scores  will  be 
evaluated  for  their  clinical  utility. 
See  Figure  4  for  an  example 
principal  EOF  weighting  along 
with  the  composite  score  for 
patient  2234.  The  principal  EOF 
seems  to  capture  the  sympathetic 
response  with  an  increase  in 
heart  rate  and  blood  pressure. 
This  can  be  seen  in  Figure  4  from 
minute  200  to  minute  450.  During 
this  time  period  rises  in  both 
blood  pressure  and  heart  rate 
lead  to  a  rise  in  the  principal  EOF. 

This  method  can  be  extended. 
Instead  of  allowing  the  statistics 
of  the  data  to  determine  the  most 
meaningful  weightings  as 
described  above,  it  is  also 
possible  to  choose  what  should 
be  an  a  priori  meaningful 
weighting.  We  focused  on 
cardiovascular  collapse  as  well  as 
respiratory  distress.  We  leverage 
clinical  knowledge  to  identify 
cardiovascular  collapse  as  a 
scenario  where  blood  pressure  is 
falling  and  other  vitals  signs  such 
as  heart  rate  and  respiratory  rate 
are  rising.  For  example  it  is  well 
known  that  patients  headed  for 
septic  shock  start  to  have  drops 
in  blood  pressure  while  heart  rate 
increases.  In  practice  these 
subtle  changes  over  hours  can  be 


Figure  5.  EOF  Composite  Score  for  Cardiovascular  Collapse 
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A  pre-specified  weighting  corresponding  to  cardiovascular  collapse  that  is 
demonstrated  during  a  portion  of  patient  2229's  time  series.  From  top  to  bottom: 
The  cardiovascular  collapse  variable  weightings,  the  projection  of  the  weightings 
onto  the  multivariate  patient  time  series  to  create  the  CV  Index,  patient  2229's 
heart  rate  time  series,  and  patient  2229's  blood  pressure  time  series. 
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Figure  5b.  A  pre-specified  weighting  corresponding  to  respiratory  distress  that  is 
demonstrated  during  a  portion  of  patient  2247's  time  series.  From  top  to  bottom: 
The  respiratory  distress  variable  weightings,  the  projection  of  the  weightings  onto 
the  multivariate  patient  time  series  to  create  the  Resp  Index,  patient  2247's 
respiratory  rate  time  series,  and  patient  2247's  oxygen  saturation  time  series. 


7 


difficult  to  pick  up  by  clinicians  because  heart  rate  and  blood  pressure  are  typically  in  the  normal  ranges  until 
the  patient  is  on  the  verge  of  shock.  For  example  Figure  5  (top)  demonstrates  that  at  approximately  minute 
120,  the  patient's  heart  rate  rises  while  blood  pressure  rises  moderately,  followed  by  a  stabilization  of  heart 
rate  and  a  decline  in  blood  pressure.  During  both  of  these  phases,  the  CV  index  rises.  Also,  for  respiratory 
failure,  we  identify  that  scenario  by  a  decreasing  oxygen  saturation,  decreasing  respiratory  rate,  and  increasing 
heart  rate.  Figure  5  (bottom)  demonstrates  that  from  approximately  minute  2000  to  minute  4000,  patient  2247 
experienced  increasing  respiratory  distress  marked  by  a  gradually  increasing  respiratory  rate  and  a  gradually 
decreasing  oxygen  saturation.  The  RESP  Index  captures  these  changes  with  a  rising  value  over  this  time 
period. 

We  believe  there  are  several  benefits  to  this  approach  including:  1)  it  captures  clinically  relevant  changes 
across  physiologic  parameters;  2)  it  may  be  generalized  to  any  number  of  parameters  and  patterns  deemed 
important  to  monitor  for;  3)  it  presents  the  results  to  clinicians  as  an  index  that  can  be  tracked  over  time  fitting 
the  existing  cognitive  model  in  intensive  care;  and  4)  results  can  also  be  displayed  in  an  overview  format 
showing  many  patients  at  once.  During  the  cognitive  experiment  we  will  test  these  hypotheses  and  evaluate 
the  perceived  helpfulness  of  these  indices  to  clinicians  to  identify  patient  change  that  may  precede  clinical 
deterioration.  Figure  6  shows  the  iterations  of  the  overview  page  to  help  clinicians  identify  which  patients  need 
attention  first.  However,  any  results  would  need  to  be  tempered  until  clinical  studies  can  be  conducted  to 
determine  if  specific  indices  are  associated  with  clinical  conditions  and  to  determine  values  for  indices  that  may 
represent  different  categories  of  clinical  risk  that  should  be  monitored  for. 


Figure  6.  EOF  Composite  Score  Overview  Format 
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Figure  6.  The  patient  overview  page  provides  clinicians  1  hour,  12  hour,  and  24  hour  windows  for  change  on 
two  pre-specified  and  two  unspecified  EOF  clinical  indices.  We  will  test  in  a  cognitive  experiment  whether 
clinicians  are  able  to  identify  patients  that  need  attention  first  using  this  approach. 
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SEPSIS  VISUALIZATION 

The  goal  of  the  SIRS  (Sepsis)  visualization  is  to  help  clinicians  identify  the  physiological  signs  leading  to 
sepsis.  This  is  challenging  for  clinicians  because  with  sepsis  there  are  a  series  of  5-6  variables,  sometimes 
more,  (HR,  HRV,  Respiration  Rate,  Temp,  WBC  (and  bands),  BP)  that  can  be  used  as  indicators  -  perhaps 
early  indicators  -  of  a  patient  going  into  septic  shock.  Generally,  if  two  or  more  criteria  are  met,  there  is  a 
suspected  infection. 

1 )  temperature  >38°C  or  lower  than  36°C; 

2)  heart  rate  >  90  beats/min; 

3)  respiratory  rate  >  20  breaths/min;  and 

4)  white  blood  cell  count  >  12,000/mm3  or  <  4,000/mm3  which  more  than  10%  immature  forms. 

At  Columbia,  clinicians  monitor  6  variables  (4  SIRS  criteria,  and  2  additional): 

1 )  temperature 

2)  heart  rate 

3)  respiratory  rate 

4)  white  blood  cell  count 

5)  heart  rate  variability 

6)  blood  pressure 

The  challenge  is  that  once  these  criteria  are  met,  it  is  often  the  case  that  the  patient  is  severely  ill.  It  is  possible 
that  we  can  help  to  detect  the  early  emergence  of  sepsis  through  supporting  clinicians  in  seeing  early  trends  in 
SIRS-related  variables.  The  most  difficult  part  (for  clinicians)  of  detecting  (early  detection)  that  a  patient  is 
potentially  going  into  septic  shock  is  in  seeing  the  slight  trending  of  multiple  variables  at  one  time.  Not  a  single 
variable  spiking,  but  instead,  it  is  the  trend  of  all  variables.  The  early  trend  of  these  variables  that  we  want  to 
support  is  that  temp,  HR,  RR,  and  WBC  go  up  and  HRV  and  BP  go  down.  Typically,  these  variables  change 
close  to  the  same  time. 

For  clinicians,  the  challenge  is  the  monitoring  for  these  slight  trends  in  multiple  variables  are  multifaceted: 

1 )  Monitoring  for  the  emergence  semi-rare  events  with  weak  evidence  is  challenging. 

2)  The  variables  that  the  clinicians  are  monitoring  are  not  all  located  in  the  same  place. 

3)  It  is  easy  to  see  many  of  the  variables  on  the  patient  monitor  (HR,  RR,  Temp),  however  that  monitor 
only  provides  the  last  10  seconds  of  data.  Longer  trends  are  not  shown  here. 

4)  Rate  of  change  of  individual  variables  is  not  easily  found 

5)  Rate  of  change  of  multiple  variables  requires  integration  in  the  clinician  or  nurses  head. 


Figure  7.  Initial  Mockup  to  identify  SIRS  and  sepsis 
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In  the  initial  mockup  the  circle  (Figure  7)  was  not  actually  meant  to  be  read  (no  individual  variables  or 
numbers).  We  believed  that  showing  an  increasing  size  (and  color  and  position)  could  be  used  to  easily 
indicate  to  clinicians  that  SIRS  is  emerging.  The  clinician  or  nurse  can  then  dig  deeper  into  the  actual  numbers 
and  notice  how  those  numbers  are  changing  over  time.  The  circle  is  simply  an  alerting  mechanism.  There  are 
several  reasons  why  we  wanted  a  simple  user  interface.  In  this  case  two  variables  trend  down  while  four  trend 
up  when  a  patient  is  becoming  septic.  Secondly  this  Ul  is  new  and  therefore  we  simply  wanted  to  grab  the 
clinician’s  attention  and  direct  it  to  a  potential  event  as  evidenced  by  these  SIRS  variables  changing  in  a  bad 
direction.  It  would  then  be  up  to  the  clinician  to  dig  into  the  trends  and  order  more  tests  (if  necessary).  The 
objective  in  this  version  of  the  mockup  was  to: 

•  Present  all  6  variables  in  a  single  location 

•  Present  longer  trends  of  all  variables 

•  Show  their  individual  values  at  time  steps  of  15-minutes. 

•  Color  code  the  individual  values  to  indicate  how  close  they  are  to  normal  or  alarm  stages. 

As  we  explored  this  design  further  however  we  changed  it  significantly  from  the  original  mockup.  The  change 
in  design  (Figure  8)  was  prompted  by  several  concerns  with  the  original  design  including: 

1 .  The  presentation  of  the  individual  variables  and  numbers  would  not  reduce  clinician  cognitive  load. 

2.  Clinicians  would  have  to  trend  many  hours  of  data  to  determine  if  changes  were  occurring,  again 
potentially  not  positively  impacting  clinician  cognitive  load. 

3.  The  current  design  maximizes  the  effect  of  a  making  a  single  comparison  between  the  current  status 
and  past  status  and  should  be  more  effective  at  reducing  cognitive  load. 

4.  The  current  design  still  presents  all  6  variables  in  a  single  location  and  allows  the  evaluation  of  longer 
trends  of  all  variables. 


Figure  8.  Stage  2  of  Sepsis  Mockup 
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The  design  continued  to  evolve  in  the  fourth  quarter.  We  incorporated  a  shock-index  graph.  The  shock-index 
is  a  metric  that  has  been  shown  in  studies  to  be  helpful  in  identifying  patients  at  high  risk  for  sepsis.  A  number 
of  changes  were  added  (Figure  9): 
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1.  Addition  of  the  shock  index.  The  shock  index  is  a  derived  value  that  can  be  used  to  indicate  potential 
sepsis  severity.  Completed  first  version  of  the  shock-index  graph.  Whether  the  y-axis  should  be  linear  or 
logarithmic  is  an  area  of  further  investigation. 

2.  We  reworked  the  observation  graphs  and  the  shock-index  graph  so  all  use  the  same  underlying  time-series 
graph.  This  was  necessary  to  facilitate  a  single  cursor  across  all  the  time-series  charts  (shock-index  and 
observation  graphs). 

3.  Normalization  of  the  sepsis  indicator:  The  sepsis  indicator  is  a  normalized  value.  The  normalization 
requires  some  tweaking  to  ensure  that  it  can  pick  up  increases  to  sepsis 

4.  Visualization  of  timelines  requires  additional  input  to  show  relative  increases  in  numbers  as  well  as  the 
numbers  themselves. 

5.  Changed  the  "change"  chart  to  use  a  red-gray-green  color  scheme.  The  color  selection  will  be  centralized 
so  that  the  "change"  chart,  the  patients  overview,  and  possible  others  use  the  same  "change-based" 
semantic  classifications  and  colors. 

6.  Warning  areas  for  specific  parameters  are  highlighted  in  yellow  and  red  based  on  parameter  specific 
thresholds.  The  EOF  indices  will  have  a  pop-up  showing  the  bar  graph  to  help  with  interpretation.  The 
design  allows  for  variables  to  be  added  or  removed  without  requiring  changes  to  the  interface. 


Figure  9.  Sepsis  Visualization  Tool 
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The  PREDICT  interface  displays  meaningful  information  about  changes  in  patient  state.  Clockwise  from  top  left 
is  patient  information,  current  risk  levels  for  physiological  systems,  an  integrated  view  of  the  current  Septic 
Shock  Index,  and  the  bottom  timeline  graph  displays  trends  and  allows  drilling  into  patient  history. _ 


INTRACRANIAL  PRESSURE  MANAGEMENT 

The  goal  and  purpose  of  the  intracranial  pressure  management  interactive  visualization  is  to  determine  an 
‘ideal’  target  for  cerebral  perfusion  pressure  that  will  maintain  adequate  perfusion  of  the  brain.  The  clinical  task 
is  to  determine  an  ‘ideal’  target  for  cerebral  perfusion  pressure  that  will  maintain  adequate  perfusion  of  the 
brain  (e.g., 6'8).  Figure  10  displays  data  from  a  single  patient  and  highlights  the  complexity  of  interpreting 
patient  monitoring  data  without  visualization  as  the  relationship  between  cerebral  perfusion  pressure  (CPP) 
and  brain  oxygen  tension  is  dependent  on  end  tidal  C02  concentrations.  When  end  tidal  C02  is  below  30  the 
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patient  appears  to  have  relatively  intact  cerebral  autoregulation,  which  is  to  say  that  vascular  systems  that 
regulate  cerebral  blood  flow  to  a  constant  flow  rate  are  functioning  properly  when  CPP  is  greater  than  80 
mmHg.  Note  the  locally  weighted  regression  line  is  essentially  flat  when  CPP  is  greater  than  80  mmHg,  but 
steadily  declines  below  that  pressure.  In  contrast,  when  end  tidal  C02  is  above  30  we  observe  cerebral 
autoregulation  failure  when  brain  oxygen  tension  increases  as  cerebral  perfusion  pressure  increases.  These 
relationships  are  impossible  to  determine  without  tools  to  visualize  high  resolution  physiologic  data,  which  are 
largely  not  available  at  the  bedside. 


Figure  10:  Interpretation  of  multimodal  data  to  identify  cerebral  perfusion  pressure  targets 
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The  figure  below  (Figure  1 1 )  presents  the  prototype  of  the  ICP  tool. 


Figure  11.  Intracranial  Pressure  Management  Visualization 
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REAL-TIME  VISUALIZATION  DEMONSTRATION 

We  have  started  to  develop  a  working  real-time  demo  that  incorporates  the  principles  of  the  three 
visualizations.  Chaos  theory  describes  the  behavior  of  attractors  in  dynamical  systems  such  as  in  human 
physiology.  In  practice  physiology  varies  around  a  physiologic  set  point  until  the  system  is  perturbed  to  a  new 
set  point.  These  ranges  can  be  identified  utilizing  financial  time  series  support  and  resistance  formulas  (Figure 
12). 

Figure  12.  Identification  of  Short-Term  Normal  Range  of  Parameters 
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Assessment  of  cerebral  autoregulation  using  the  pressure  reactivity  index  (PRx)  is  a  common  application  of 
multimodal  neuromonitoring  data.  We  have  developed  a  demonstration  real-time  web  application  to  enable 
clinicians  to  assess  cerebral  autoregulation  status  and  identify  potential  patient  state  changes  in  real-time  at 
the  bedside  (Figure  13).  We  have  uploaded  a  movie  that  demonstrates  this  functionality  (Appendix  1). 
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In  addition  we  have  modified  the  basic  design  to  accommodate  multiple  data  streams  and  visualizations  from 
different  tools  being  developed.  In  Figure  14,  we  have  included  elements  from  both  the  overview  and  sepsis 
visualization  prototypes  into  a  single  tool  (Figure  14).  This  can  be  viewed  using  simulated  data  on 
http://demos.imateapot.net/neurovis/bedview.php.  This  is  hosted  on  a  low  traffic  web  server  that  can  only 
support  one  to  two  simultaneous  connections,  so  please  do  not  widely  distribute  this  link  at  this  time. 


Figure  14.  Incorporation  of  Visualization  Elements 


COGNITIVE  EXPERIMENT 

Phase  2  of  the  project  has  begun  and  its  objective  is  to  conduct  a  cognitive  experiment  to  test  whether  the 
visualization  improves  clinician  decision  making.  We  are  recruiting  physicians  at  three  levels  of  training  to 
participate  in  this  study.  This  includes  faculty  that  have  completed  residency  and  fellowship  in  neurocritical 
care,  as  well  as  fellow  and  resident  trainees.  To  date  we  have  consented  three  faculty  and  five  residents  to 
participate.  Seven  clinical  fellows  have  not  yet  been  approached  to  participate. 

Overview 

Physicians  are  presented  a  clinical  vignette  with  data  and  asked  to  identify  the  frequency  of  physiologic  events, 
assess  overall  patient  status,  identify  tests  and  treatments  they  would  order,  and  report  the  likelihood  that 
clinical  events  have  occurred  or  will  occur  in  the  next  24  hours. 

Design 

The  mode  of  physiologic  data  presentation  (visualization  versus  text)  and  clinical  event  type  (sepsis  versus 
other  events)  are  the  two  independent  variables.  Several  hypotheses  are  tested:  1)  The  speed  and  accuracy 
of  responses  are  better  when  physiologic  data  is  presented  in  a  visualization  versus  a  text  spreadsheet;  2) 
Clinical  events  are  more  likely  to  be  detected  in  advance  of  onset  in  the  visualization  versus  text  condition;  3) 
sepsis  detection  specifically  is  improved  when  using  data  visualization.  To  test  these  hypotheses  clinical 
vignettes  were  generated  from  actual  clinical  cases  of  3  patients  experiencing  sepsis  and  3  experiencing 
clinical  events  other  than  sepsis.  A  vignette  was  created  for  each  patient  for  the  day  the  clinical  event 
occurred,  and  for  each  of  the  2  days  leading  up  to  the  event  day.  Each  patient  day  is  presented  in  both 
visualization  and  text  form  for  a  total  of  36  (6  patients  x  3  patient  days  x  2  conditions)  vignettes. 

The  two  most  likely  threats  to  the  internal  validity  of  the  study  were  clinicians  recognizing  that  clinical  cases 
were  presented  in  both  text  and  visualization  conditions,  and  that  clinical  events  were  likely  to  occur  on  the  3rd 
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day  of  the  case.  Specifically,  if  unaltered,  the  same  history  of  present  illness  would  be  presented  six  times, 
and  the  clinical  exam  and  labs  for  each  patient  day  would  be  presented  with  physiologic  data  in  text  and 
visualization  conditions.  To  control  for  these  potential  internal  validity  threats,  we  generated  six  versions  of  the 
history  of  present  illness  and  two  versions  of  the  clinical  exam  and  labs  information.  This  was  done  in  a 
manner  that  made  it  unlikely  that  the  case  would  be  recognized  while  maintaining  the  clinical  meaning  of  the 
presentation.  In  the  instruction,  participants  are  told  that  they  would  be  presented  “a  series  of  similar  cases”  to 
reduce  questions  about  the  similarity  of  patients.  In  total,  clinicians  are  presented  36  vignettes  that  appear  to 
be  created  from  36  different  patients,  although  in  reality  come  from  6  patients.  Figure  15  shows  an  example  of 
these  modifications  for  the  same  patient  day. 

Figure  15.  Patient  Vignette  Modified  for  Visualization  and  Text  Conditions. 

Version  1 

History  of  Present  Illness: 

67  year  old  woman  with  diabetes  on  daily  baby  aspirin  who  presented  to  an  outside  hospital  with  neck  pain  and  altered  mental  status.  CT  imaging  showed  subarachnoid 
hemorrhage.  She  was  then  transferred  to  CUMC  where  repeat  imaging  showed  intraventricular  hemorrhage  (Hunt  Hess  3,  Fisher  grade  4,  modified  Fisher  grade  4),  and 
mild  hydrocephalus.  She  was  admitted  to  the  Neuro  ICU  for  further  monitoring  where  digital  subtraction  angiography  was  negative  for  aneurysms,  and  an  EVD  was 
piaced. 

Post-bleed  day  9 

Physical  exam: 

*  Lungs  with  bilateral  crackles  in  bases 

*  Normal  cardiac  exam 

*  Abdomen  soft,  nontender,  nondistended,  bowel  sounds  present 

*  Mildly  erythematous  H  upper  extremity,  mild  swelling 

Neuro  exam: 

*  Asleep,  eyes  closed;  responds  to  noxious  stimulation  only,  follows  commands 

*  PERRL 

*  Localizes  with  both  UE  to  noxious  stimuli;  withdraws  with  both  LE  to  noxious  stimuli 


Version  2 

Patient  Vignette 

History  of  Present  Illness: 

64  year  old  woman  with  HTN  and  DM2  who  was  admitted  to  the  CPMC  NICU  with  Hunt  &  Hess  Grade  3r  Fisher  Grade  4,  modified  Fisher  grade  4  subarachnoid 
hemorrhage  with  early  hydrocephalus.  CT  angiography  showed  no  aneurysms  nor  evidence  of  radiographic  vasospasm,  and  no  aneurysm  was  seen  on  conventional 
angiogram.  On  arrival  to  the  NICU,  she  underwent  EVD  placement  for  hydrocephalus. 

Post-bleed  day  13 


Physical  exam: 

*  Eyes  open  spontaneously;  constant  stimulation  is  necessary  to  keep  awake 

*  Oriented  to  date;  perseverates 
■  Cranial  nerves  intact 

*  Moves  all  4  extremities  equally  and  antigravity 

Imaging: 

*  GXR  with  worsened  pulmonary  edema 

24  Hour  Labs: 

*  CSF  extracted  from  EVD:  G68  PI 27  616  WBC,  693  RBC 

The  survey  is  estimated  to  take  approximately  three  to  four  hours  to  complete.  The  survey  is  broken  down  into 
three  sessions  of  12  patient  vignettes.  Study  participants  may  also  stop  between  cases  within  a  survey. 
Balancing  was  used  to  generate  the  list  of  patient  case  days  contained  in  each  survey  (Table  1).  Lastly, 
participants  take  a  10  minute  survey  to  evaluate  the  overview  visualization  in  which  clinicians  are  presented  a 
single  visualization  and  asked  to  identify  the  three  patients  they  would  round  on  first  and  last. 

Procedure 

Physicians  are  presented  a  clinical  vignette  and  are  provided  the  history  of  present  illness,  clinical  exam 
information  and  standard  laboratory  tests  for  the  current  patient  day.  Physiologic  data  is  then  presented  in 
either  a  text  spreadsheet  similar  to  what  is  used  in  an  electronic  medical  record  or  in  the  visualization.  With  this 
information  participants  are  asked  to  identify  the  frequency  of  physiologic  events,  assess  overall  patient  status, 
identify  tests  and  treatments  they  would  order,  and  report  the  likelihood  that  clinical  events  have  occurred  or 
will  occur  in  the  next  24  hours.  Responses  to  each  question  are  timed  allowing  speed  and  accuracy  to  be 
determined.  A  case  used  in  the  experiment  is  shown  Figures  16-21 . 
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Table  1.  Balancing  presentation  of  case  days  in  three  surveys  by  condition 


Control  Condition  Patient 

Patient  Day 

Visualization 

Text 

N01 

-2 

Survey  1 

Survey  2 

N01 

-l 

Survey  2 

Survey  3 

N01 

Event (0) 

Survey  3 

Survey  1 

N02 

-2 

Survey  2 

Survey  3 

N02 

-1 

Survey  3 

Survey  1 

N02 

Event (0) 

Survey  1 

Survey  2 

N03 

-2 

Survey  3 

Survey  1 

N03 

-1 

Survey  1 

Survey  2 

N03 

Event (0) 

Survey  2 

Survey  3 

Sepsis  Condition 

Patient 

Patient  Day 

Visualization 

Text 

SOI 

-2 

Survey  3 

Survey  1 

SOI 

-1 

Survey  1 

Survey  2 

SOI 

Event (0) 

Survey  2 

Survey  3 

S02 

-2 

Survey  2 

Survey  3 

S02 

-1 

Survey  3 

Survey  1 

S02 

Event (0) 

Survey  1 

Survey  2 

S03 

-2 

Survey  1 

Survey  2 

S03 

-1 

Survey  2 

Survey  3 

S03 

Event (0) 

Survey  3 

Survey  1 

Figure  16.  Initial  patient  vignette 


Patient  Case 


Patient  Vienette 

History  of  Present  Illness: 

64  year  old  woman  with  HTN  and  DM2  who  was  admitted  to  the  CPMC  NICU  with  Hunt  &  Hess  Grade  3,  Fisher  Grade  4,  modified  Fisher  grade  4  subarachnoid 
hemorrhage  with  early  hydrocephalus.  CT  angiography  showed  no  aneurysms  nor  evidence  of  radiographic  vasospasm,  and  no  aneurysm  was  seen  on  conventional 
angiogram.  On  arrival  to  the  NICU,  she  underwent  EVD  placement  for  hydrocephalus. 

Post-bleed  day  13 


Physical  exam: 

•  Eyes  open  spontaneously;  constant  stimulation  is  necessary  to  keep  awake 

•  Oriented  to  date;  perseverates 

•  Cranial  nerves  intact 

•  Moves  all  4  extremities  equally  and  antigravity 

Imaging: 

•  CXRwith  worsened  pulmonary  edema 

24  Hour  Labs: 

•  CSF  extracted  from  EVD:  G68  P127  618  WBC,  893  RBC 


136 

101 

15 

3.3 

23 

1.1 

41.  Please  copy  the  following  text  into  firefox  to  view  the  patient  data.  To  confirm  that  you  have  the  correct  data  for  this  patient,  please  enter  the  last  observed  heart  rate 
measurement  for  this  period.  The  other  questions  will  appear  once  the  correct  value  is  entered.  If  you  have  any  problem  please  ask  for  help. 

file:///O:/NICU/Shared%20Projects/Michael%20Schmidt/Experiment/PREDICT%20-%20vl0/PREDICT/index.html#/patients/90331* 


Next 
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Figure  17.  Patient  data  presented  in  visualization 
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Figure  18.  Patient  data  presented  in  text  spreadsheet 
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Figure  19.  Survey  questions:  identify  events,  patient  status,  tests,  and  orders 

42.  Which  events  occurred  duri  ng  the  time  period  shown?  * 


Never  occurred 

<  i  hour 

1-  3  hours 

3  - 12  hours 

>  12  hours 

Unavailabi  e/Unknown 

Tachycardia 

o 
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Fever 
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o 
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o 
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o 
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o 
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0 
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o 

o 

Hypertension 
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o 

o 

0 

o 

o 

Hypotension 

o 

o 

o 

o 

o 

o 

ICPefevation 
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o 

43.  How  has  the  patient  CHANGED  during  the  time  period  shown  (i.e,  has  the  physiology  gotten  better  or  worse  during  the  day]?  * 

Much  Improved  Improved  No  Change  Worse  Much  Worse  Unavailable/Unknown 

Patient  Status  0  O  O  O  O  0 

44.  Which  tests  (if  any)  would  you  order  for  this  patient  based  on  the  information  provided?  * 

45.  Are  there  any  therapeutics  that  you  would  order?  * 
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Figure  20.  Likelihood  of  clinical  events  in  last  24  hours  and  next  24  hours 


46.  Given  the  information  available  to  you,  what  is  the  likelihood  that  this  patient  has  experienced  any  of  these  conditions  within  the  LAST  24  hours?  * 
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Next 


REMAINING  WORK  AND  CHALLENGES 

Conducting  the  cognitive  experiment,  analysis  of  its  data,  and  reporting  upon  the  results  is  the  remaining  work 
to  be  completed.  Data  collection  is  in  progress  and  will  6-10  weeks  to  complete.  The  project  continues  to 
move  forward,  though  more  slowly  than  anticipated,  and  will  be  completed  without  the  need  for  additional 
funding. 

FUTURE  PLANS 

Results  from  the  cognitive  experiment  will  provide  feedback  needed  to  refine  the  visualization  tools  that  we 
have  developed.  In  our  cognitive  work  analysis  we  also  identified  a  need  for  a  patient  visualization  that 
provides  an  overview  of  all  critical  systems  for  the  patient,  including  labs  and  medications.  The  workflow 
progression  would  be  to  use  the  overview  visualization  developed  in  this  proposal  to  identify  which  patient 
needs  to  be  evaluated.  The  next  visualization  would  then  provide  a  broad  overview  for  an  individual  patient, 
including  elements  that  have  been  developed  in  this  proposal,  as  well  as  additional  elements  that  help  the 
clinician  identify  clinical  events  and  alarm  status,  the  physiologic  state  of  organ  systems  including  labs,  and 
current  medications.  The  clinicians  would  identify  areas  of  concern  and  could  then  drill  down  into  the  sepsis  or 
ICP  management  visualizations  to  facilitate  a  specific  treatment  decision. 
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Additionally,  we  have  begun  to  create  multiple  mockups  for  other  interactive  visual  support  tools  to  help 
clinicians  with  other  cognitive  and  collaborative  challenges  (See  uploaded  powerpoint  presentation).  We  have 
also  connected  our  web-based  tool  to  a  real-time  data  feed,  but  there  are  limitations  to  this  approach  in  terms 
of  how  much  information  a  web  browser  can  hold  without  experiencing  slowdown.  There  is  a  need  to  study 
more  intensively  what  the  right  combination  of  static  and  live  data  should  be  used  to  support  visualization 
applications.  These  ideas  are  worthy  of  further  study  and  as  such  our  proposed  plans  for  next  steps  include: 

Aim  #1:  Refine  the  current  prototype  -  Based  on  our  experiment  results  the  first  step  would  be  to  further 
refine  the  designs  and  add  a  new  patient-specific  summary  page.  We  have  included  a  powerpoint  presentation 
(Appendix  A)  with  mockups  that  expand  on  many  of  these  ideas. 

Aim  #2:  Transition  the  current  prototype  to  real-time  system  -  This  would  involve  data  model  development 
to  ensure  the  appropriate  data  syntax  and  semantics  are  represented  for  the  real-time  PREDICT  system.  Then 
the  IMEDS  middleware  would  be  implemented  to  provide  the  appropriate  services  while  the  IMEDS  API  is 
developed  to  support  the  PREDICT  system  requirements  and  ICE  compliance. 

Aim  #3:  Deploy  PREDICT  system  to  Columbia,  Emory,  and  UCLA  -  Expanding  to  include  three  different 
institutions  is  critical  to  understanding  generalized  and  specific  clinician  needs  and  IT  design.  Quantitatively 
assessing  how  these  differences  affect  the  innovative  decision  support  tool  and  clinical  benefits  (related  to 
patient  outcomes)  is  important  for  future  adoption  by  other  clinical  environments. 

Aim  #4:  Evaluate  PREDICT  system  at  three  sites  to  determine  impact  on  results  of  patient  outcomes 
and  paths  to  those  outcomes  -  An  evaluation  including  a  structured  survey  of  clinicians’  experience  in  ease 
of  data  acquisition,  quality  of  decision-making,  and  time  saved  (or  wasted),  and  patients’  clinical  outcomes. 


KEY  RESEARCH  ACCOMPLISHMENTS 

•  Devised  methods  to  both  filter  and  interpolate  physiological  data. 

•  Created  physiologic  indices  for  clinically  meaningful  variation  among  several  parameters  using  a 
version  of  empirical  orthogonal  functions. 

•  Applied  the  above  physiologic  indices  to  create  a  patient  overview  visualization  to  help  clinicians 
identify  patients  that  are  changing  and  inserted  these  indices  into  the  sepsis  specific  decision  support 
visualization. 

•  Created  a  sepsis  identification  visualization  tool  to  help  clinicians  identify  patients  headed  for  septic 
shock. 

•  Generated  a  visualization  tool  to  help  clinicians  assess  intracranial  pressure  and  generate  patient 
specific  physiologic  set-points  for  goal  directed  therapy. 

•  Demonstrate  a  real-time  data  feed  to  a  visualization  tool  that  contains  elements  from  each  of  the  three 
prototype  visualizations. 

REPORTABLE  OUTCOMES 

•  Schmidt,  J.  M.,  Irvine,  J.  M.,  &  Miller,  S.  (in  press).  Data  Visualization.  In  M.  DeGeorgia  &  K.  Loparo 
(Eds.),  Neurocritical  Care  Informatics:  Translating  Data  into  Bedside  Action:  Springer. 

•  Schmidt,  J.  M.,  Doerner,  J.,  Sow,  D.,  Perotte,  A.,  &  Mayer,  S.  BEDSIDE  APPLICATION  FOR  THE 
ASSESSMENT  OF  CEREBRAL  AUTOREGULATION  AND  PATIENT  STATE  CHANGES.  Neurocritical 
Care,  2013.  19,  S205. 

•  Schmidt,  J.  M.  (2013,  November).  Streaming  Analytics  Case  Study  in  Columbia  University  Neuro-ICU 
Paper  presented  at  the  EME  Streaming  Analytics  Symposium,  Chicago,  Illinois. 
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CONCLUSION 


In  the  intensive  care  unit  environment  clinicians  are  under  a  constant  state  of  information  overload.  Patients  in 
need  to  medical  attention  are  not  identified  in  a  time  to  treat  them  beneficially,  and  mismanagement  from  a  lack 
of  understanding  of  patient  occurs  too  frequently.  These  problems  lead  to  lower  quality  patient  care  resulting 
in  more  secondary  complications,  less  than  optimal  outcomes  for  patients,  and  longer  ICU  length  of  stay 
increasing  healthcare  costs.  Results  from  our  cognitive  work  analysis  confirmed  the  need  for  improvements  in 
patient  information  presentation  to  facilitate  clinical  decision  making.  We  identified  the  need  for  a  patient 
overview  screen  that  contains  data  from  bedside  monitors,  electronic  health  record,  and  laboratory  systems. 
This  type  of  visualization  was  not  within  the  original  scope  of  work  and  remains  an  important  unmet  need.  The 
three  visualization  prototypes  that  we  did  develop  are  meant  to  help  clinicians  on  a  variety  of  important  clinical 
tasks  including:  1 )  prioritizing  patients  most  in  need  of  medical  attention,  2)  identifying  patients  at  high  risk  for 
sepsis,  and  3)  managing  intracranial  pressure  crises  and  identifying  patient  specific  physiologic  targets.  A 
cognitive  experiment  will  be  conducted  in  the  next  period  to  determine  what  works  and  what  does  not  work 
regarding  our  visualization  designs;  this  will  allow  firmer  conclusions  to  be  drawn  and  will  support  the  next 
phase  of  development  for  medical  data  visualization  and  decision  support  in  the  intensive  care  unit 
environment. 

Several  steps  need  to  be  completed  to  move  towards  a  deployable  clinical  decision  support  tool.  A  new  phase 
of  visualization  development  is  needed  informed  by  the  findings  of  the  cognitive  experiment.  Over  the  last  year, 
each  visualization  evolved  substantially  and  this  process  should  continue.  The  visualization  tools  need  to 
optimized  for  real-time  deployment,  either  fed  by  a  real-time  data  stream  as  we  have  demonstrated  here  or  fed 
from  a  real-time  file  structure.  Deploying  and  evaluating  the  visualization  tools  to  multiple  sites  is  a  crucial  step 
in  this  development.  This  will  provide  a  framework  for  general  deployment  and  provide  an  opportunity  to 
evaluate  these  designs  clinically  to  determine  if  in  fact  decision  support  tools  improve  clinician  decision  making 
resulting  in  less  secondary  complications,  shorter  ICU  length  of  stay,  and  improvements  in  long-term  outcome. 
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APPENDICES 

•  Powerpoint  presentation  containing  mockup  visualizations  for  future  work. 
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